Evaluating Information Dispersal Algorithms

The explosion in data acquisition and storage has led to the emergence of data-intensive applications that are used to process enormous quantity of information using methods such as the MapReduce paradigm. Data-Intensive Distributed File Systems (DI-DFS) have been designed to support these kinds of applications. These large-scale storage systems require faulttolerance mechanisms to handle failures, which are a norm rather than an exception when working at a scale that will shortly reach exascale. A new trend among large-scale systems is the implementation of information dispersal algorithms, called erasure codes. The overhead introduced by the encoding and decoding can be a limiting factor for the integration at large-scale. This work compares two different approaches of erasure code computing on GPU and CPU. Our work in erasure coding serves as the foundation for the next step: integrating an information dispersal algorithm that eventually outperforms current state-ofthe-art approaches in DI-DFSs.